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Abstract 

The purpose of this study was to adapt Reflective Thinking Scale to Turkish and investigate its validity and 
reliability over a Turkish university students’ sample. Reflective Thinking Scale (RTS) is a 5 point Likert scale 
(ranging from 1 corresponding Agree Completely, 3 to Neutral, and 5 to Not Agree Completely), purposed to 
measure reflective thinking in habitual actions, understanding, reflecting and critical reflecting subscale levels. 
With the help of education area and language experts, the scale adaptation thoroughly completed and the cor¬ 
relation between the original scale and its Turkish form indicated satisfactory language equality (r=.80). During 
the 2011-2012 school years, 1413 undergraduate students, from varying departments of Faculty of Education, 
Canakkale Onsekiz Mart and Gaziosmanpasa University, were participated in this study. In order to maximize 
student participation, convenient sampling method was followed. After the item analysis, internal consistency 
of the scale and subscale items were checked through Cronbach Alpha internal consistency coefficient and 
Spearman Brown Split-Half correlation coefficient and both indicated good internal consistency on the scale 
and the subscale levels. The scale also demonstrated sufficient test-retest reliability over the application of the 
scale in two weeks and two months intervals. The construct validity is established through a confirmatory and 
an exploratory factor analyses and the four factor structure, suggested in the original study, was confirmed by 
both analyses. The concurrent validity was also confirmed through the scale's satisfactory Pearson correlation 
coefficient with Critical Thinking Scale. 
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Producing reflective thinkers is one of the main goals 
of a constructivist approach to education and also 
one of the primary objectives of Deweys pragma¬ 
tism. Dewey (1933) described reflective thinking as 
“active, persistent, and careful consideration of any 
belief or supposed form of knowledge in the light of 
the grounds that support it and the further conclu¬ 
sion to which it tends”. He also stated that reflective 
thinking adds meaning to the experience through its 


reorganization and reconstruction and leads the way 
for further goals requiring more comprehensive tasks. 
Reflective thinking requires the learner to achieve 
the learning goals and turn into the behavior. Later, 
learner evaluates his/her own performance and has 
a perception of the progress he/she made, makes a 
discussion about the product, this way it will be pos¬ 
sible to benefit from that experience for the future 
occurrences. A deed can be counted as successful 
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to the extent that its doer reflects his/her education 
towards life experiences. For this purpose, a teacher 
needs to promote students through helping them with 
homework assignments and demonstrations. Pollard 
(2002) emphasized that teacher reflection has become 
increasingly important for classroom practitioners, 
and both Tripp (2003) and Pollard stated that reflec¬ 
tive thinking is a cyclic process. According to Ennis 
(1987), reflective thinking is a component of the crit¬ 
ical thinking process. Tan and Goh (2008) stated that 
reflection links the past, the present, and/or the future. 

Kember and his colleagues (Kember et al., 2000; 
Kember, McKay, Sinclair, & Frances, 2008) explained 
reflective thinking in four dimensions as Habitual 
Action, Understanding, Reflection, and Critical Re¬ 
flection. They also concluded that this classification 
can be used to evaluate the appropriateness of stu¬ 
dents’ writings, compositions, dairies, and their an¬ 
swers to open ended questions to reflective thinking. 

Habitual Actions: These are the actions, learned be¬ 
fore and mastered through frequent use and started 
to be performed automatically. Using a keyboard, 
riding a bike, driving a car can be given as examples 
of this level. 

Understanding: According to Mezirow (1991), 
thoughtful action makes use of the knowledge, with¬ 
out attempting to appraise that knowledge (cited in 
Kember et al., 2000). Our learning that has taken 
place in schools; Mezirows “book learning” is clas¬ 
sified under this category. Although first Kember 
et al. (2000) attempted including all categories of 
Blooms taxonomy in the Understanding sub scale 
of the questionnaire, later they realized that the scale 
had poor psychometric properties; therefore they 
have decided to focus on Understanding or Compre¬ 
hension in order to increase reliability and validity 
through concentrating on a narrower construct. 

Reflection: There are many definitions of reflec¬ 
tive learning and how it should be. Kember et al. 
(2000) based their article on Mezirows definition. 
They were also influenced by Boyd and Fales (1983) 
and Boud, Keogh, and Walker (1985) who empha¬ 
size the importance of experiences in the reflection 
process. As cited by Kember et al. (2000), Mezirow 
describes reflection as validity testing. 

Reflection involves the critique of assumptions 
about the content or process of problem solving. 
The critique of premise or presupposition pertains 
to problem posing as distinct from problem solving. 
Problem posing involves making a taken-for-grant- 
ed situation problematic, raising questions regard¬ 
ing its validity (Mezirow, 1991, p. 105). 


According to Boyd and Fales, reflective learning is the 
process of internally examining and exploring an issue 
of concern, triggered by an experience, which creates 
and clarifies meaning in terms of self, in a changed 
conceptual perspective (cited in Kember et al., 2000). 

Critical Reflection: This is a higher level reflective 
thinking. Mezirow calls it “premise reflection”. In 
this level, learners become aware of why he/she 
perceive, think, feel, or act (cited in Kember et al., 
2000). Kember et al. (2000) stated that it would not 
be easy to transform perspectives. Dewey (1933) 
also underlines the importance of critical thinking 
by suggesting not reaching a hasty conclusion with¬ 
out examining all the possibilities. Reflective think¬ 
ing is a much discussed construct and there are not 
many scales measuring. However, critical thinking 
is regarded as a close call to reflective thinking. 
Reflective Thinking Scale is developed by Kember 
and his colleagues in 2000. There is also a scale de¬ 
veloped by Semerci (2007) purposed to measure 
teacher candidates’ reflective thinking inclination. 

Although it was developed not long before, Reflec¬ 
tive Thinking Scale has been used cross culturally. 
Leung and Kember (2003) used the scale on health 
class students in Hong Kong University and found 
that students who tend to learn on surface, scored 
higher in Habitual Action level, while deep learners 
are more likely to be in Understanding, Reflection 
or Critical Reflection Levels. According to Leung 
and Kember this finding can be taken as a sign of 
convergent validity of the scale. Pahn (2007) also 
confirmed these findings as a result of his study, 
searching reflective thinking in comparison with 
self-efficacy beliefs of students. Lie (2007) also found 
that students who are in the first year of their educa¬ 
tion based on problem solving methods had lower 
critical thinking levels compared to other students 
in upper grades. There are also findings of a qua- 
si-experimental study indicating higher scores in the 
experimental group for Understanding and Critical 
Thinking Sub-scales and Higher scores for the con¬ 
trol group in Habitual Action Sub-scale (Mahardale, 
Neville, Jais, & Chan, 2008). In addition, there are 
some researches about reflective thinking and peer 
education (Loke & Chow, 2003), and learning ap¬ 
proaches (Leung & Kember). Besides Lucas and Tan 
(2006) emphasized that Reflective Thinking Scale is 
valid on social sciences field students. Dolap<;iog- 
lu (2007) and Karadag (2010) stated that reflective 
thinking levels of the Turkish teachers were found 
high. Erginel-§anal (2006) and Koksal and Demirel 
(2008) found that reflective thinking education has 
positive effects on the reflective thinking level. 
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Literature review indicated that there was not any 
attempt to adapt this scale into Turkish. Therefore, 
it would be a good addition to the literature, provid¬ 
ing researchers an instrument to measure reflective 
thinking in order to increase students’ achievement. 

Method 

Model 

This study was based on a descriptive survey meth¬ 
od. The reliability and validity of Reflective Think¬ 
ing Scale were studied from many aspects. 

Study Sample and Application 

Since the main purpose of the current study was to 
search construct validity of Reflective Thinking Scale 
over students, taking different classes by different in¬ 
structors, it was decided to carry on a big scale study. 
The Sample of the study consisted of 1413 university 
students from the Faculty of Education of Qanakkale 
Onsekiz Mart University (n=968) and Gaziosman- 
pa§a University (n=445). The sample drawn by conve¬ 
nient sampling consists of students, whose ages were 
between 18 and 37, coming from different class level. 
70% of students were girls and 30% were boys. 

In order to prove scales language equity, original form 
and its Turkish form applied to a group of 1513rd grade 
students in an English Teacher Education Program, 
with a one week interval. Test-retest reliability was also 
tried on a 147 Teacher Candidate sample in the Faculty 
of Education of Gaziosmanpa§a University. 

Measurement Instruments 

Reflective Thinking Scale: It is a 16 item 5-point 
Likert scale ranging from 5 Definitely Agree, 4 
Agree with reservation, 3 only to be used if a Defi¬ 
nite answer is not possible, 2 Disagree with reser¬ 
vation, to 1 Definitely disagree. The scale has four 
subscales which are Habitual Action, Understand¬ 
ing, Reflection, and Critical Reflection. The lowest 
possible score that can be earned from each sub 
scale is 4 and the highest is 20. The scale is very 
easy to understand and it does not take more than 
10 minutes to complete. The original validity study 
was carried out in Hong Kong University in 2000 
on 303 university student sample. The Cronbach 
Alpha coefficients for the subscales ranged between 
.62-.76. A confirmatory factor analysis affirmed the 
four factor model for reflective thinking. 

The literature review indicated that there is a need 


for adapting the Reflective Thinking Scale into 
Turkish since it is measuring an important feature 
of constructivist approach. The current study was 
intended to fill this gap in the literature. 

The Scale Adaptation into Turkish 

In the process of adaptation, the scale first trans¬ 
lated into Turkish by the authors of the study. The 
language equity of the scale was confirmed by three 
area experts who are fluent in both languages. The 
Turkish form of the scale was improved accord¬ 
ingly to these feedbacks. The final form was back 
translated by two instructors from the English De¬ 
partment. For delivering handouts before exams by 
instructors is not a common application by Turkish 
instructors, it was decided to re-write Item 9 as “As 
long as I study for the exams and remember what we 
learned in class, there is no need for me to think too 
much”. Also, Item 14 is also rewritten to imply un¬ 
derstanding to be in fine with the other three items 
in the Understanding sub scale. Later, the form was 
read by a group of graduate students in order to eval¬ 
uate its appropriateness for undergraduate students. 
The final form was re-evaluated by two area experts 
and two philosophers to decide its equality to its 
original form after the final improvements. After a 
careful review by a Turkish language expert, the scale 
adaptation process was completed. A pilot study of 
the scale on a group of 15 students indicated that all 
students completed the scale in 10 minutes most. 
The only problem with the scale, during the applica¬ 
tion process was that a few students asked what was 
meant by “this course” in the items since they were 
taking several courses. The administrator explained 
that it meant the course they were having a class 
meeting right at that moment. It was decided that 
during the application process students were better 
to be reminded what was meant by “this course” in 
the items. This will not be a problem when the scale 
is applied to measure students’ reflection levels in a 
certain course. 

The California Critical Thinking Disposition In¬ 
ventory 

The California Critical Thinking Disposition Inven¬ 
tory (CCTDI) was developed as a part of American 
Philosophy Association’s Delphi Project in order 
to measure critical thinking intentions by Facione 
and Facione (1992). The scale adapted to Turkish by 
Kokdemir in 2003 has 51 items and six subscales as 
analyticity, self confidence, inquisitiveness, maturity, 
open-mindedness, being systematical, and truth seek- 
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ing. Cronbach Alpha internal consistency for the scale 
is .88 and for the sub-scales it ranges from .61 to .78. 

Analysis 

After a routine check for unusual data points and nor¬ 
mality check for each item, the factorial structure of 
the scale was checked by an exploratory factor analysis 
through SPSS and a confirmatory factor analysis was 
performed by Lisrel. Item discrimination levels were 
searched through item-total correlation coefficients 
and upper and lower group comparisons through 
independent group t tests. Reliability checks were ac¬ 
complished by Cronbach Alpha internal consistency 
coefficient, Spearman Brown Split Half reliability, and 
test-retest correlation coefficients. 

Results 

Turkish and English versions of the scale were ap¬ 
plied in two week intervals to 151 English Depart¬ 
ment 3rd grade students. It was seen that the correla¬ 
tions ranged from .74 to .92. Therefore, it was decid¬ 
ed that the scale had adequate language equality. 

Item Analysis 

Factor loadings are all positive and over .50. A nega¬ 
tive factor loading means that the item has a negative 
relationship with the rest of the items. There were 
no reverse coded items in the scale and no negative 
loadings indicating all items have positive relation¬ 
ships with each other. Communalities ranged be¬ 
tween .36 and .59. High communalities indicate that 
the item has power in explaining the variation in 
data. Corrected item-total correlations are found be¬ 
tween .21 and .50 which is above the cut point of .20. 
Also, upper and lower group comparison indicated 
that all items had discriminating power. 

Reliability 

Reliability of the scale was attained through the 
test-retest reliability, Cronbach Alpha coefficient and 
split half correlations. Overall total scale scores, Un¬ 
derstanding, Reflective Thinking and Critical Reflec¬ 
tion Subscales had sufficient reliability. Results indi¬ 
cated low reliability with Habitual Learning Subscale. 
Considering that the scale has four items under each 
sub scale level, this can be understandable. 

Correlations between the subscales and the total 
score were higher than the correlations among the 
subscales. The low correlations among the subscales 
indicated that the scale has more than one dimen¬ 
sions (subscales). Considerably high correlations 
between the total score and subscales can be tak¬ 
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en as a sign that the subscales indeed measure the 
same domain altogether. 

Significant mean differences between the subscales 
can also indicate that subscales measure different 
domains. All of the pair-wise comparisons between 
the subscales were significant; therefore the sub- 
scales measure different parts of reflective thinking. 

Validity 

Construct validity of the scale is searched through 
an explanatory and confirmatory factor analysis. In 
addition, criterion validity of the scale is established 
through its correlation with California Critical 
Thinking Scale. 

Exploratory Factor Analysis: Kaiser Mayer Olkin 
Measure found .83 indicating that sample was ade¬ 
quate to run a factor analysis. Bartlett’s Chi Square 
test (2352.11, p= .001) was significant, indicating 
that data was appropriate to draw factors. Principal 
component analysis was run with a Promax rota¬ 
tion. According to Tabachnick and Fidel (1996) in 
a principal component analysis when the sub-scales 
have high correlations, one of the orthogonal ro¬ 
tations should be used. According to Kaiser Rule 
(Kaiser, 1960) and Cattell’s Scree Plot (Cattell, 1966) 
there were four factors. According to Kline (1994) 
40% of the total variation explained by factors, can 
be taken as the minimum acceptable explained vari¬ 
ance level. Sixteen items together explained 53% 
of variation in data. The same items appeared un¬ 
der the same headings as in the original study. In 
order to accept an item under a dimension, factor 
loading value .40 and above were taken as a cut off 
value (Stevens, 1996). Item loadings were all positive 
and ranged between .76 at highest and .54 at lowest. 
Communalities were between .41 and 63. Reflection 
had the highest eigenvalue with 3.78 and explained 
24% of the variance. Reflection followed by Criti¬ 
cal Reflection with an eigenvalue of 2.11 and 13% 
of the variation. There was one item (Item 1) with a 
.21 item-total correlation level. According to §encan 
(2005), with a sample size over 400 cases, .20 and 
above item total correlation values can be regarded 
enough. Ebel (1965) also suggests that an item with 
an item-total correlation value between .20 and .30 
can be used if it does not have an alternative and 
when it is necessary (cited in Turgut, 1990, p. 270). 

Confirmatory Factor Analysis: Through Lisrel 
8.54 (Joreskog & Sorbom, 2003), it was searched 
whether the data support the advocated four fac¬ 
tor model for the scale, suggested by its developers. 
Proposed four factor model compared to one factor 
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model. Since model-data suitability was affected 
by sample size, X 2 /df division is used. Any value 
below five indicates a good fit. Goodness of fit in¬ 
dex (GFI), Normalized Standard Fit Index (NCFI), 
Comparative Fit Index (CFI), Normative Compar¬ 
ative Fit Index (NCFI), Adjusted Goodness of Fit 
Index (AGFI), and Root Mean Square Error Ad¬ 
justed (RMSEA) were used to decide which model 
is a better fit to the data. According to Anderson 
and Gerbing (1984) when NNFI and CFI values are 
close to 1, this indicates a good fit, this is true also 
for GFI and AGFI indexes. RMSEA value is expect¬ 
ed to be less than .08 (Byrne, 2001; Hu & Bentler, 
1998), NNFI, CFI, GFI and AGFI values are also 
expected to be close to 1 for a good fit. 

According to table 1, it was seen that single factor 
model has the poorest properties as it was in the 
original study. Four-factor model indicated a good 
fit with the X 2 /df value less than 5 and all the fit in¬ 
dexes close to 1. 


Table 1. 

Fit Indexes for Different Models 

Model 

X 2 /df 

GFI 

AGFI 

NNFI 

CFI 

RMSEA 

1. Single 
Factor Null 
Model 

10.49(104) 

.80 

.74 

.77 

.80 

.13 

2. Four 
Factor 
Model 

4.48 (98) 

.93 

.90 

.92 

.93 

.07* 

p<.001** 


Concurrent Validity of the Scale 

Concurrent validity of the scale was searched 
through its correlation with California Critical 
Thinking Scale. Results indicated that total scores 
for two scales had a satisfactory correlation. 

Results and Discussion 

Reflective Thinking Scales Turkish Form confirmed 
the four factor structure of the original scale. EFA 
and CFA results both confirmed the construct va¬ 
lidity of the scale. The low correlations among the 
subscales indicate that subscales do not overlap. 
The item analysis indicated that all items were psy- 
chometrically sound. Cronbach Alpha and Spear¬ 
man Brown Split-Half correlation coefficients indi¬ 
cated that the scale had good internal consistencies 
both in the scale and subscale level, all of them near 
or above the critical level of .70, set by Nunnally 
and Bernstein (1994). The scale also demonstrated 
sufficient test-retest reliability over the application 
of the scale in two weeks and two month intervals. 
The scales’ concurrent validity was also confirmed 
through its satisfactory correlation with Critical 
Thinking Scale (Kokdemir, 2003). 
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